I built a one-parameter model that gets 100% on ARC-AGI-2
I built a model that has only one parameter and gets 100% on ARC-AGI-2, the million-dollar reasoning benchmark that stumps ChatGPT. Using chaos theory and some deliberate cheating, I crammed every answer into a single number 260,091 digits long.
Intro
"When a measure becomes a target, it ceases to be a good measure" - Charles Goodhart
In July 2025, Sapient Intelligence released their Hierarchical Reasoning Model (HRM) and the world went crazy. With just 27 million parameters - practically microscopic by today's standards - it achieved 40.3% on ARC-AGI-1, a notoriously difficult AI benchmark with over a million dollars in prize money. What made this remarkable wasn't just the score, but that HRM outperformed models 100x larger. In October came the Tiny Recursive Model, obliterating expectations yet again. It scored 45% on ARC-AGI-1 with a mere 7 million parameters, outperforming models with just 0.01% of their parameters.
Naturally, I wondered: how small can we go?
So I built a one parameter model that scores 100% on ARC-AGI-2.
This is on ARC-AGI-2, the harder, newer version of ARC-AGI-1. The model is not a deep learning model and is quite simple:
||[
\begin{align*}
f_{\alpha, p}(i)
& :=
\sin^2 \Big(
2^{i p} \arcsin(\sqrt{\alpha})
\Big)
\tag{1}
\end{align*}
||] where ||(x_i||) is the ||(i\text{th}||) datapoint and ||(\alpha \in \mathbb{R}||) is the singe trainable parameter. (||(p||) is a precision hyperparameter, more on this later.) All you need to get 100% on ARC-AGI-2 is to set ||(\alpha||) to
alpha=0.76059909954446680055519182637612404564952882608728406710332547552128446698293425221832975998898637236392205440417981617949548706406387030215295746057916672370704613139993102981520849196296541741232578489544689528846131754394446902690191625779049723017787574644995645998191196995652299872827305403826756449151125077232068806629551684890541253359837329652493258169800543968350639741142817814171771642248712635256943748361193985927343404950453292524945805333934593051464400368944775599614089386320090273869491583657873220635267085752381431568997047115636182977812657757703361060373156080128282582144284187660985488238464936573395047573286639794573061683106383780627206026056150656838337588742335170670163421547280089023114092731598196680570418789874360527785012620300750930571954145011629243254516400206074041792073533356634146282504823968716433512687354617522447535226537565807236324564511110530062555438093437991795253214900545506687774536460148324596924222765480221641621437626327952134561001137023114376121540714156077998979093947571760303967431735253560946726570431340478430446602897672186508688227963721583043604075440291847665238470622066363109790582849794551747780485489595489733526399388459089360532382024529983473279000118880550004458935134473467937295051711283017650338558799208485823692220113215500266839592398203647032543908678681690879298999981442507542699041885524438564023588682378780079564157156045238634045398695077157108768698635811519895727218480343642120199553556922633561513832993044251371733496713714038013023954617181980966126968599387312241882727248751576631562411547791297718419456595377424699163855402216397498164401130962968790610811366593327400604013568472112487874809243504048659211852460086449909371510922263800983013104904627190443959378599852522275841718258587191805316977713078922928979606542956843709965037688753815417491928819404900356691314870543851739837241147833212652252936159211863971744398094927646740679199631006426174223380727505335307167988032644482607296973649537963162183440473216940605032558101492556872708356280796276003648091179335379035258289160415302359363858900666153704691764931925925534406272390086086237329041610751547532019997897493078792907406941578146573475206151497477485229315269180219574632906375189695908689730230717428916617325031998926780420806061386666046378583550071837247219466671697801843275332629753429308619047895535595684965817408085518888469910885027747090610603268324756078382749082990796132294651696881962928843629666725462682052451754753210899245133102912862225695572223573065210237545360047657039153983266072141586295117635668640223145638447889078773869374500518526813523569610350738113091270685978352989525552843022947752887419320518566118681697782148121899491792256309190853432950405647536031737543390421102237554138964154770691717834238142835295903088016342454100425770493220937550307457687387407409126952330334735420567375549795107901955493785165495953409323894060830435620570206635849734662458737137483566337649534813334036348785228898195429450462856437144411867635374055751693375071803133807123831194651089909562997282413021342896561111606763292751884999306433844319748614157717565114832200109899469975816243813734116195852267016018802776752909623986709057339404407174549479917623262292240692002948370839150438273618393548860906396549033845355030052330412431992551370306302513601841256455352161676868724062742771044714295698520709295655308658285122710482792611122497388344593857474560674622871831336277084951405000971113340387563469498883887914139569735935236194021553028097485092302193506412589956409691851635642983987557063969527137748846851575268282918423430404714076185770651479593680730853234713962389176638219778886764674683880048309236507035373824040232233029653571112335890752198564060769684218360193945775160795150387853632496549813389083360477029277096925068714425757948063374241049937951042803950342033445371303446297868003510919059361392419647184606546262096418104251841412863845536203737205379212842289342292425511612761797538648763406643439484569320659994291100675343273494669834294803734900203737013417100118038378603369901381043977052704473311759416532482038269394632551113175893258270851602265446447153690360333251382016326750069857718712564112021845023970168381605698323370567169959382223112364162500269668256178549619453546360694675626436892488960735152429417700094916671647123637211445155968709499281655596987120695031551539821456765527088897416280303321025535004008268548802003627233782548010446344265286141915222903410738787642286337933982028810804614620125176438395819871970310118074213659161270060303320147119740836883503451883920252341235864479602200143227174386696981900286498662348078264892352177818224004514752719165951094725795533280934772262927043111481494055926359424235970372517366815655300580013381348371759046046757473872400568034434821922993214630315067626807995383783042059170832893457171236288972797216424484243452006314419875885003142696934614100686515162788694105289772813597152167946168786411249023743671931125014764497751746904121124768439753373523227956202640430097973548435354421886542664469609014595268032114191836055029851062320977153773519695423715145616656763470440175259897425518305743905458086381771427323082710416295272874800283981121797120387330285020531180553330873675973570329780940699892742659404700958312842654586541380065209576923861953494953244980589742400910631653909675098367373824932315128757405422051078783149635657283367454784236047812606123857121673521141126766180764298752559700776860395728257996327358401478326259259581959799308176211406304030132166284562177971066260407527995949612839043009242449428647277062867168836292110234255156931233274729663376277524208114248267584966459255180803428756039291103438018453955386531039465367863091803176115996981580478418519447947178777991641391483748965509969338227280424957069644781368246006940789745400817158831463264620202428306475038428611078677821771576377491124868731029308476997399021578735567393066015642577983921537588625044630586559382290440763847114321212536067418322983439160609583436016154595162039972480980008332228395199864804623366206046287202278433837789586772432779453559606141046902293501407679939567136846571571044081627160686123718818620365067743226481257586704522628234395828674169980644108757404872121179081250052105606280632371512636304875614875331673091940145417773319286941057345915626255651256729555241877906619662397348792553925012003504573694580532849897401247362118653720468736926707714573232436483875436639567948796062100481620396560271087641548481214783088169791478644511421343721492599932808338944442497931996750503707235112639922385857006307374476783681421229116265172974206771525132488764702906253909587073474212412935113386687248996231488463264927114176176606782589023428441986080284180720406177115214452516286888557188451460815210496326837237342356812415596479229526963796184718103077582460237516373032222533111717015708360009181166186690893761125871866251795808680194454675535629991894668957901068786133880222894562528879618149779133631559413294408508925660066552175982654999623437731390458892269796380923510697846674851511524236432846015453841617878844151495980039563576277468710559559504734256804555726188729215422981580796997679996581269745921191688088036166307731239199049156530789760311766256833468490055725653931258165633614413547709825554792482801243007903177214026017284086802906815362569255783313807919992331182775130760898847381443827489130336791007435996494988663449256356596407576032037042868049377909937976049387851484409322778626321293144713204042513873194660105633232156901765941969751014202828772356314444902651251269300918644946117200428049344749431282107095654700352461861678601623592735881068759036403473880535373166022301274570514276804616802944429547142667545675143272142233592045135545713589472102792508441269682856084437121442734453596431049608610221712876070764938673298309988081317006986451486585275957574350389443844496652865513850933422222636435646808265687072317060243536366215919702941680133258636467236343604460913909796315104891664793520407151458107200330377393189336723045525364105197618678446866094876776848855145935421068170006587555400643512387023828224954504917720764100636200870509420992454158888516314733399981887759362941973842609558522566136501600005644156223359471562815428537113376652186934562870261429731322862610015089495425001449434545412102225919924752872028502296356828005239492882941451727218054065987947481435028418402731328638226359122882508844898430328384505492681038657793082287020532873923483089778902549365648252634899913502774099888711917849515178703156937257827721341028407183500575227930534413117949974122387752675340845477960971741691484023286722182936686890443041556517488002348174742780584563327505255949041342212396547936708911843336538611672001659958722287011009586434792648803519492611640809174833433699160739691715399816525889574764376903304639463002159954653568953732482568617001962812913542623874561154246160760917505884553729053860769351026228431265217434495664888734090602779849054111646975721102984229708675731050327792231950360663655358601924534192157063431462199691679554980955937647441387119458510455722292254930602000085017298529943018391539707128115102269103636130391850166105780428390896371390288202528948283189290089607075758795759210654736664310179145536021185898118071999824826475858149963386764096920104575668186308592692256147392895871823073456433440211252581338448477237978225232410058902652561385066844893333838747728803852386387116824664646824817016036916255256755661471496476513928593768702338738478831308164254414245655118209131884370225087353169977898716573363356970996025441219355836090236496022132391110897791070642850110249138078715353935306599126752466095648124899980927786228296878603245860532827519988760643395988696926590071732290612978947566092627148724282313454981871542909915965930571941721513364476016868409145994772071192490287994289503839837245950902627121243405749749445336817
p=8
and you'll get a perfect score on ARC-AGI-2! (Feel free to scroll but only the first 10,000 digits of ||(\alpha||) are shown.)
This number is 260,091 digits long and is effectively god in box, right? One scalar value that cracks one of the most challenging AI benchmarks of our time.
Sounds pretty impressive, right?
Unfortunately, it's complete nonsense.
There is no learning or generalization. What I've really done here is train on test and then use some clever mathematics from chaos theory to encode all the answers into a single, impossibly dense parameter. Rather than a breakthrough in reasoning, it's a very sophisticated form of cheating.
This one-parameter model is a thought experiment taken seriously. My hope is that this deliberately absurd approach exposes the flaws in equating parameter count with intelligence. But this also exposes a deeper issue at play. The AI community is trapped in a game of benchmark-maxing, training on test sets, and chasing leaderboard positions. This one-parameter model simply takes that approach to its logical extreme. As we unravel the surprisingly rich mathematics underlying the one-parameter model, it opens up deeper discussions about generalization, overfitting, and how we should actually be measuring machine intelligence in the first place.
Let me show you how it works.
ARC-AGI
"Intelligence is measured by the efficiency of skill-acquisition on unknown tasks. Simply, how quickly can you learn new skills?" - ARC-AGI creators
Too many benchmarks measure how good AI models are at a particular skill rather than measuring how good they are at acquiring a new skill. AI researcher François Chollet created The Abstraction and Reasoning Corpus for Artificial General Intelligence (ARC-AGI-1) to fix this. ARC-AGI-1 measures how well AI models can generalize to unseen tasks. It consists of problems that are trivial for humans but challenging for machines. More recently, ARC-AGI-2 was released as a more challenging follow up to ARC-AGI-1. This blog will focus on ARC-AGI-2.
What makes ARC-AGI-2 different from typical benchmarks?
Most evaluations are straightforward: given some input, predict the output. ARC-AGI-2, however, is more complicated. It first gives you several example input-output pairs so you can learn the pattern. Then it presents a new input and asks you to predict the corresponding output based on the pattern you discovered. This structure means that a single ARC-AGI-2 task consists of:
||(0||) and ||(9||) where ||(1 \leq n, m \leq 30||) . To display the grid, we simply choose a unique color for each integer. Let's look at an example:
- several example input-output pairs
- a question input
- a question output
n x m matrix (list of lists) of integers between 
Here, we see several grids, each with a bunch of colored cells. Most cells are black (0), some are red (2), and some are light blue (8). Each column shows an input-output pair.
The first three columns are example input-output pairs that demonstrate the pattern. The fourth column, separated by the vertical black line, is the actual question: given this new input, what should the output be? Here we show the question output as a source of ground truth. The model is suppossed to predict this and is never given access to it.
Now, how do you solve this specific task?
Looking at the examples, each grid contains exactly two shapes: one red and one blue. The pattern is straightforward: translate the red shape in a straight line toward the blue shape until they touch (but do not overlap). The blue shape remains stationary. The resulting configuration -- red shape adjacent to blue shape -- is the output.
In Example 1, the red shape sits in the upper left and the blue square in the mid-right. Translating the red shape horiztonally to the right, it slides until it reaches the blue square, resulting in the example output. The question follows the same logic. In the question input the red shape sits in the middle of the grid and the blue shape is in the mid-left. Translating the red shape horizontally to the left, it slides until it reaches the blue shape. Looking at the question output, we can verify that this indeed matches the question output.
Here is another task.
Looking at the examples, each input contains exactly 3 diagonal lines, each a single solid color. The pattern is to repeat these 3 colors cyclically across the entire output grid in diagonal stripes, creating a repeating checkered pattern. Whether the input's 3 diagonals appear consecutively or not doesn't matter, they repeat every 3 diagonal positions throughout the output.
In Example 1, the input shows three consequtive diagonal stripes: blue, red, and yellow. The output tiles this sequence repeatedly -- blue diagonal, red diagonal, yellow diagonal —- cycling through all 3 colors across the full grid. Loking at the question, the input also contains three diagonal lines in blue, red, and yellow, but they're in a different order and do not appear all next to each other. Still, the output repeats these three colors cyclically in diagonal stripes, filling the entire grid. The pattern cycles every 3 diagonals: red, blue, yellow over and over again. This produces the a different output than Example 1 because the order of the three colors is different.
There are hundreds of tasks like this in ARC-AGI-2. Solving each task requires deducing new patterns and generalizing to unforeseen tasks, something it is quite hard for the current crop of AI models.
Even the world's best models struggle on ARC-AGI-2, all scoring under ||(50\%||) . ||(45.1\%||) but costs a staggering ||(\$77.16||) per task. ||(\$7.14||) per task but only solving ||(18.3\%||) of tasks. Many other frontier models -- Claude, Grok, and Deepseek can't even crack ||(20\%||) . In contrast, humans get ||(100\%||) of questions right. That's why there exists a ||(\$1,000,000||) competition to open source a solution to ARC-AGI-2. It's that difficult.
Gemini 3 Deep Think (Preview) has the highest score of GPT-5 Pro is much more efficient, costing The HRM Drama
In July, HRM was released. It is a fascinating model, inspired by the human brain with "slow" and "fast" loops of computation. It gained a lot of attention for it's amazing performance on ARC-AGI-1 despite its tiny size of 27M parameters.
HRM scored 40.3% on ARC-AGI-1 while SOTA models like o3-mini-high and Claude-3.7-8k scored 34.5%, and 21.2% respectively (back in July 2025). It beat Anthropic's best model by nearly ~2x! Similarly, it outperformed o3-mini-high and Claude-3.7-8k on ARC-AGI-2, but be warned that the ARC-AGI-2 the scores are so low that they are more much suspectable to noise.
The results almost seemed to be too good to be true. How can a tiny 27M parameter model from a small lab be crushing some of the world's best models, at a fraction of their size?
Turns out, HRM trained on test:
In their paper, the HRM authors admitted to showing the model "example pairs in the training and the evaluation sets". The evaluation set here refers to the public eval set of ARC-AGI-1! This sounds like training on test!
On github, the HRM authors clarified that they only trained on the examples of the public eval set, not the questions of the public eval set. This "contraversy" set AI twitter on fire [1, 2, 3 4 5] ! Does this actually count as "training on test"? The HRM authors never actually trained on the the questions used to measure model performance, just the examples associated with them. However, these have very similar distributions...
Ultimately the ARC-AGI organizers accepted HRM's submision and the concsensus on Twitter was that it's actually completely allowed to train on the examples of the public eval set. But buried in a GitHub thread, HRM's lead author, Guan Wang, made an offhand comment that caught my attention:
||(40.3\%||) on ARC-AGI-1, what happens with complete leakage? If we train on the actual test questions, not just test examples, can we hit ||(100\%||) ? Can we do it with even fewer parameters than HRM (27M) or TRM (7M)? And can we do it on the more challenging ARC-AGI-2 instead of ARC-AGI-1? How far can we push this?
"If there were genuine 100% data leakage - then model should have very close to 100% performance (perfect memorization)." - Guan WangThat line stuck with me. If partial leakage gets you
Chaos Theory
"Chaos is what killed the dinosaurs, darling." - J.D. in Heathers
My goal was simple: create the tiniest possible model that achieves perfect performance on ARC-AGI-2 by blatantly training on the public eval set, both the examples and questions. We would deviate from HRM's acceptable approach (training on just the examples of the public eval set) and enter the morally dubious territory of training on the examples and questions of the public eval set.
Now, the obvious approach would be to build a dictionary - just map each input directly to its corresponding output. But that's boring and lookup tables aren't nice mathematical functions. They're discrete, discontinuous, and definitely not differentiable. We need something else, something more elegant and interesting. To do that, we are going to take a brief detour into the world of chaos theory.
||(\mathcal{D}||) is a simple one-dimensional chaotic system defined as
||[
\begin{align}
\mathcal{D}(a)
&=
(2a) \bmod 1
&
\mathcal{D}: [0, 1] \to [0, 1].
\tag{2}
\end{align}
||] It takes in any number between 0 and 1, doubles it, and throws away the whole number part, leaving just the fraction. That's it.
Note: As far as I know, Steven Piantadosi pioneered this technique in One parameter is always enough. Yet I first heard of it through Laurent Boué's paper Real numbers, data science and chaos: How to fit any dataset with a single parameter. This paper is really a gem due its sheer creativity.In chaos theory, the dyadic map
In chaos theory, we often study the orbit or trajectory of a chaotic system, the sequence generated by applying the chaotic map to itself over and over again. Starting with some number ||(a||) , we apply our map to get ||(\mathcal{D}(a)||) , and again to get ||(\mathcal{D}(\mathcal{D}(a))||) , and so on and so forth. Let
||[
\begin{align*}
\mathcal{D}^k(a)
& :=
\underbrace{(D \circ ... \circ D)}_{k}(a) = (2^k a) \mod 1
\tag{3}
\end{align*}
||] mean we apply the dyadic map ||(k||) times to ||(a||) . What does the orbit ||((a, \mathcal{D}^1(a), \mathcal{D}^2(a), \mathcal{D}^3(a), \mathcal{D}^4(a), \mathcal{D}^5(a))||) look like?
- If
||(a = 0.5||) , the orbit is||((0.5, 0.0, 0.0, 0.0, 0.0, 0.0)||) . - If
||(a = 1/3||) , the orbit is||((0.333, 0.667, 0.333, 0.667, 0.333, 0.667,)||) - If
||(a = 0.431||) , the orbit is||((0.431, 0.862, 0.724, 0.448, 0.897, 0.792)||)
| Iterations | Decimal | Binary | Observation |
|---|---|---|---|
| 0 | Original number | ||
| 1 | First bit of | ||
| 2 | First two bits of | ||
| 3 | First three bits of | ||
| 4 | First four bits of | ||
| 5 | First four bits of |
- If
||(a = 0.5||) , we get the orbit||((0.5, 0.0, 0.0, 0.0, 0.0, 0.0)||) because||(\text{bin}(a) = 0.100000...||) and after discarding the first bit, which is a||(1||) , we are left with all zeros. - If
||(a = 1/3||) , we get the orbit||((0.333, 0.667, 0.333, 0.667, 0.333, 0.667)||) because||(\text{bin}(a) = 0.010101...||) , an infinite sequence of bits alternating between||(1||) and||(0||) . When the bits start with a 0, we get||(0.010101...||) which is||(1/3 = 0.333||) in decimal. And when the bits start with a||(1||) ,||(0.10101...||) , we get||(2/3 = 0.667||) in decimal.
The Dyadic Map As An ML Model
"When I grow up, I'm going to be a realboyML Model" - the Dyadic Map if it were staring in Pinacoi
We've discovered something remarkable: each application of ||(\mathcal{D}||) peels away exactly one bit. But here's the question: if the dyadic map can systematically extract a number's bits, is it possible to put information in those bits in the first place? What if we encode our dataset into a number's bits (
model.fit) and then use the dyadic map as the core of a predictive model, extracting out the answer bit by bit (model.predict)? In other words, can we turn the dyadic map into an ML model?A Worked Example
Suppose our dataset contains the three numbers we saw before
||[
\mathcal{X}
=
\{x_0, x_1, x_2\}
=
\{0.5, 1/3, 0.431\}.
||] Let's convert each number to binary and look at the first ||(p=6||) binary digits for simplicity:
||[
\mathcal{B}
=
\{b_0, b_1, b_2\}
=
\{ \text{bin}_6(x_0), \text{bin}_6(x_1), \text{bin}_6(x_2)\}
=
\{0.100000, 0.010101, 0.011011\}
||] where the function ||(b_i = \text{bin}_p(x_i)||) converts decimal numbers to ||(p||) -bit binary numbers. Now comes the clever part: we glue these binary strings together, end to end:
||[
b
=
0.
\underbrace{100000}_{b_0}
\underbrace{010101}_{b_1}
\underbrace{011011}_{b_2}
||] and convert this binary string back to decimal
||[
\alpha = \text{dec}(b) = 0.50522994995117188
||] The number ||(\alpha||) is carefully engineered so that it is a decimal number whose bits contain our entire dataset's binary representation. That's right: we've just compressed our entire dataset into a single decimal number! We only have one parameter, not billions here! This is a very simple, stupid version of ||(\alpha = \text{model.fit}(\mathcal{X})||) .
But here's the question: given ||(\alpha||) , how do we get our data ||(\mathcal{X}||) back out? How do we do ||(\tilde{x}_i = \text{model.predict}(\alpha)||) ? This is where the dyadic map becomes our extraction tool.
Step 1. Trivially, we know the first 6 bits of ||(\alpha||) contains ||(b_0||) .
||[
\begin{align*}
\alpha
&=
0.50522994995117188
\\
\text{bin}(\alpha)
&=
0.\underbrace{100000}_{b_0}\underbrace{010101}_{b_1}\underbrace{011011}_{b_2}
=
0.100000010101011011
\end{align*}
||] So we'll just record the first ||(6||) bits of ||(\alpha||) to get ||(b_0||) .
||[
\begin{align*}
b_0
&=
\text{bin}(\alpha)_{0:6}
=
100000
\end{align*}
||] If we convert this number ||(b_0||) back to decimal, we'll recover our original data, up to the first ||(6||) digits of precision.
||[
\begin{align*}
\tilde{x}_0
&=
\text{dec} ( b_0 )
=
0.500000
\end{align*}
||] Now from ||(\alpha||) we've extracted the prediction ||(\tilde{x}_0 = 0.500000||) which matches exactly the ||(0||) th sample of our dataset ||(x_0 = 0.5||) .
Step 2. To predict the next number, ||(\tilde{x}_1||) , remember that each application of ||(\mathcal{D}||) strips away the leftmost binary digit. So
||[
\begin{align*}
D^6(\alpha)
&=
0.334716796875
\end{align*}
||] strips away the first ||(6||) bits of ||(\alpha||) , which just removes ||(b_0||) , and leaves us with ||(b_1, b_2||)
||[
\begin{align*}
\text{bin}(D^6(\alpha))
&=
0.\underbrace{\hspace{1cm}}_{b_0}\underbrace{010101}_{b_1}\underbrace{011011}_{b_2}
=
0.010101011011
\end{align*}
||] Like before, we'll then record the first ||(6||) bits of ||(D^6(\alpha)||) to get ||(b_1||)
||[
\begin{align*}
b_1
&=
\text{bin}(D^6(\alpha))_{0:6}
=
010101
\end{align*}
||] and convert ||(b_1||) back to decimal to get ||(\tilde{x}_1||)
||[
\begin{align*}
\tilde{x}_1
&=
\text{dec} (b_1)
=
0.328125
\end{align*}
||] Here our prediction ||(\tilde{x}_1 = 0.328125||) is slightly off from the true value ||(x_1 = 1/3||) due to the limits of ||(6||) -bit precision. If we'd have more digits of precision and increase ||(p||) , ||(\tilde{x}_1||) would be closer to ||(x_1||) .
Step 3. To get the next number, ||(b_2||) , apply ||(\mathcal{D}||) another 6 times to remove a total of ||(12||) bits from ||(\alpha||) ,
||[
\begin{align*}
D^{12}(\alpha)
&=
0.421875
\end{align*}
||] which strips off ||(b_0, b_1||) and leaves us with just ||(b_2||)
||[
\begin{align*}
\text{bin}(D^{12}(\alpha))
&=
0.\underbrace{\hspace{1cm}}_{b_0}\underbrace{\hspace{1cm}}_{b_1}\underbrace{011011}_{b_2}
=
0.011011
\end{align*}
||] Like before, we'll then record the first ||(6||) bits of ||(D^{12}(\alpha)||) to get ||(b_2||)
||[
\begin{align*}
b_2
&=
\text{bin}(D^{12}(\alpha))_{0:6}
=
011011
\end{align*}
||] and convert ||(b_2||) back to decimal to get ||(\tilde{x}_2||)
||[
\begin{align*}
\tilde{x}_2
&=
\text{dec} (b_2)
=
0.421875
\end{align*}
||] Notice again that our prediction ||(\tilde{x}_2 = 0.421875||) is slightly off from the true value ||(x_2 = 0.431||) due to the limitations of ||(6||) -bit precision.
Let
||[
\begin{align*}
\tilde{\mathcal{X}}
&=
\big \{\tilde{x}_0, \tilde{x}_1, \tilde{x}_2 \big\}
=
\big \{ 0.500000, 0.328125, 0.421875 \big \}
\end{align*}
||] be the predictions made by our strange dyadic model. If everything is correct, our predicted dataset ||(\tilde{\mathcal{X}}||) should perfectly equal our original dataset ||(\mathcal{X}||) up to the first ||(p||) bits.
These 3 steps are summerized in the table below.
In decimal, we go from ||(\alpha = 0.50522994995117188||) to ||(\mathcal{D}^6(\alpha) = 0.33471679687500000||) and then to ||(\mathcal{D}^{12}(\alpha) = 0.42187500000000000||) . Although this pattern looks completely random, we are shifitng bits with superb precision. This is anything but random.
Think about what we've accomplished here. We just showed that you can take a dataset compress it down to a single real number, ||(\alpha||) . Then, using nothing more than repeated doubling and truncation via ||(\mathcal{D}||) , we can perfectly recover every data point ||(\tilde{\mathcal{X}}||) up to ||(p||) bits of precision. The chaotic dynamics of the dyadic map, which seemed like a nuisance, turns out to be the precise mechanism we need to systematically access the desired information.
| Iteration | |||||
|---|---|---|---|---|---|
The Algorithm
The algorithm itself is deceptively simple once you see the pattern:
||(\alpha||) with ||(np||) bits of precision that contains our entire dataset. We can now discard ||(\mathcal{X}||) entirely.
||(g: [0, 1]^n \to [0, 1]||) that compresses the dataset and a decoder function ||(f: \overbrace{[0, 1]}^{\alpha} \times \overbrace{\mathbb{Z}_+}^{p} \times \overbrace{[n]}^i \to [0, 1]||) that extracts individual data points:
||[
\begin{align*}
\alpha
&=
g(p, \mathcal{X}) := \text{dec} \Big( \bigoplus_{x_i \in \mathcal{X}} \text{bin}_p(x_i) \Big)
\tag{4}
\\
\tilde{x}_i
&=
f_{\alpha, p}(i) := \text{dec} \Big( \text{bin}_p \Big( \mathcal{D}^{ip}(\alpha) \Big) \Big)
\end{align*}
||] where ||(\oplus||) means concatenation.
The precision parameter ||(p||) controls the trade-off between accuracy and storage efficiency. The larger ||(p||) is, the more accurately our encoding, but the more storage it takes up. Our error bound is
||[
|\tilde{x}_i - x_i | < \frac{1}{2^p}
||] because we don't encode anything after the first ||(p||) bits of precision.
What makes this profound is the realization that we're not really "learning" anything in any conventional sense. We're encoding it directly into the bits of a real number, exploiting it's infinite precision, and then using the dyadic map to navigate through that number and extract exactly what we need, when we need it.
From this perspective, the dyadic map resembles a classical ML model where the encoder ||(g||) acts as ||(f||) acts as
Encoding Algorithm: Given a datasetThe result is a single, decimal, scalar number||(\mathcal{X} = \{x_0, ..., x_{n-1}\}||) where||(x_i \in [0, 1]||) , encode the dataset into||(\alpha||) :
- Convert each number to binary with
||(p||) bits of precision||(b_i = \text{bin}_p(x_i)||) for||(i=0, ..., n-1||) - Concatenate into a single binary string
||(b = b_0 \oplus ... \oplus b_{n-1}||) - Convert to decimal
||(\alpha = \text{dec}(b)||)
Decoding Algorithm: Given sample indexMathematically, we can express these two algorithms with an encoder function||(i \in \{0, ..., n-1\}||) and the encoded number||(\alpha||) , recover sample||(\tilde{x_i}||) :
- Apply the dyadic map
||(\mathcal{D}||) exactly||(ip||) times||(\tilde{x}'_i = \mathcal{D}^{ip}(\alpha) = (2^{ip} \alpha) \mod 1||) - Extract the first
||(p||) bits of||(\tilde{x}'_i||) 's binary representation||(b_i = \text{bin}_p(\tilde{x}'_i)||) - Covert to decimal
||(\tilde{x}_i = \text{dec}(b_i)||)
model.fit() and the decoder model.predict().Applying Some Makeup
"You don’t want to overdo it with too much makeup" - Heidi Klum
How do we go from the ugly, discontinuous decoder function
||[
f_{\alpha,p}(i) := \text{dec} \Big( \text{bin}_p \Big( \mathcal{D}^{ip}(\alpha) \Big) \Big)
||] to that beautiful function I promised you at the start of the blog
||[ f_{\alpha, p}(i)
=
\sin^2 \Big(
2^{i p} \arcsin^2(\sqrt{\alpha})
\Big)
?
||] In this section we will "apply makeup" to the first function to get it looking a bit closer to the second function. We will keep the same core logic but make the function more ascetically pleasing. To do this, we will need another one-dimensional chaotic system, the logistic map.
Logistic Map
The logistic-map at ||(r=4||) on the unit interval is defined as
||[
\begin{align*}
\mathcal{L}(a_L)
&=
4 a_L (1 - a_L)
&
\mathcal{L}: [0, 1] \to [0, 1]
\tag{6}
\end{align*}
||] which seems quite different than the familiar dyadic map
||[
\begin{align*}
\mathcal{D}(a_D)
&=
(2 a_D) \mod 1
&
\mathcal{D}: [0, 1] \to [0, 1]
\end{align*}
||] One is a bit-shifting operation, the other is a smooth parabola that ecologists use to model population growth. (Note: previously ||(a||) was the input to the dyadic map but from now on ||(a_D||) will be the input to the dyadic map to differentiate it from ||(a_L||) , the input to the logistic map.)
What does the logistic orbit ||((a_L, \mathcal{L}^1(a_L), \mathcal{L}^2(a_L), \mathcal{L}^3(a_L), \mathcal{L}^4(a_L), \mathcal{L}^5(a_L))||) look like? Similar or different to the dyadic orbit ||((a_D, \mathcal{D}^1(a_D), \mathcal{D}^2(a_D), \mathcal{D}^3(a_D), \mathcal{D}^4(a_D), \mathcal{D}^5(a_D))||) ?
||[
\begin{align*}
a_L = \phi(a_D) &= \sin^2(2 \pi a_D)
&
\phi: [0, 1] -> [0, 1]
\tag{7}
\\
a_D
=
\phi^{-1}(a_L)
&=
\frac{1}{2 \pi} \arcsin (\sqrt{a_L})
&
\phi^{-1}: [0, 1] -> [0, 1]
\tag{8}
\end{align*}
||] We can map any ||(a_L||) to an ||(a_D||) and any ||(a_D||) to an ||(a_L||) .
- If
||(a_L = a_D = 0.5||) , the logistic orbit is||((0.5, 1.0, 0.0, 0.0, 0.0, 0.0)||) while the dyadic orbit is||((0.5, 0.0, 0.0, 0.0, 0.0, 0.0)||) . Although the dyadic orbit is just all zeros, the logistic orbit actually has||(1.0||) as the second number in the orbit. - If
||(a_L = 1/3||) , the logistic orbit is||((0.333, 0.888, 0.395, 0.956, 0.168, 0.560)||) while the dyadic orbit is||((0.333, 0.667, 0.333, 0.667, 0.333, 0.667)||) . While the dyadic orbit repeats in a simple pattern, the logistic orbit is seemingly patternless. - If
||(a_L = 0.43085467085||) , the logistic orbit is||((0.431, 0.981, 0.075, 0.277, 0.800, 0.639)||) while the dyadic orbit is||((0.431, 0.862, 0.724, 0.448, 0.897, 0.792)||) . Both orbits here seem totally random and chaotic, but each in their own way.
Looking at the plot, the function ||(\phi||) has a period of 1, meaning it repeats the same values every time ||(a_D||) increases by ||(1||) . This periodicity is crucial because it allows us to drop the modulo operation from the dyadic map ||(\mathcal{D}(a_D) = (2 a_D) \mod 1||) when transforming from the dyadic space to the logistic space. Formally,
||[
\begin{align*}
\phi(a_D \mod 1) = \phi(a_D)
\tag{9}
\end{align*}
||] which will be important later on. To go back and forth between the dyadic and logistic maps, we apply ||(\phi||) to the output ||(\mathcal{D}||) and get ||(\mathcal{L}||) ; we can also apply ||(\phi^{-1}||) to the input ||(a_L||) to get ||(\mathcal{D}||) . Mathemtically,
||[
\begin{align*}
\mathcal{L}(a_L)
&=
\phi(\mathcal{D}(a_D))
\\
\mathcal{D}(a_D)
&=
\mathcal{L}(\phi^{-1}(a_L))
\end{align*}
||] Here ||(\phi||) takes us to the logistic space and ||(\phi^{-1}||) takes us back to the dyadic space. This is astonishing! ||(\phi||) is just a sin wave squared and with a period of one. It's inverse, ||(\phi^{-1}||) is even weirder looking with an ||(\arcsin||) . But somehow these functions allow us to bridge the two maps ||(\mathcal{D}||) and ||(\mathcal{L}||) !
Moreover, ||(\phi||) and ||(\phi^{-1}||) perfectly relate every single point in the infinite orbits of ||(\mathcal{D}||) and ||(\mathcal{L}||) :
||[
(a_D, \mathcal{D}^1(a_D), \mathcal{D}^2(a_D), ...) = (a_L, \mathcal{L}^1(\phi^{-1}((a_L)), \mathcal{L}^2(\phi^{-1}((a_L)), ...)
||] or it can be expressed as
||[
(a_L, \mathcal{L}^1(a_L), \mathcal{L}^2(a_L), ...) = (a_D, \phi(\mathcal{D}^1(a_D)), \phi(\mathcal{D}^2(a_D)), ...)
||] depending on if we want to be natively using the coordinate system of ||(\mathcal{D}||) or ||(\mathcal{L}||) . What appears as chaos in one coordinate system manifests as the exact same chaos in the other, no matter how many iterations we apply. Mathematically, this suggests something stronger:
||[
\begin{align*}
\mathcal{L}^k(a_L)
&=
\phi(\mathcal{D}^k(a_D))
\tag{10}
\\
\mathcal{D}^k(a_D)
&=
\mathcal{L}^k(\phi^{-1}(a_L))
\tag{11}
\end{align*}
||] Think of these two orbits existing in parallel universes with ||(\phi||) and ||(\phi^{-1}||) acting as the bridges between ||(\mathcal{D}||) and ||(\mathcal{L}||) .
We can now revist the dyadic and logistic orbits when ||(a_D = a_L = 0.431||) .
||(\phi||) and ||(\phi^{-1}||) .
- When
||(a_D = a_L = 0.431||) , we know the dyadic orbit is||((0.431, 0.862, 0.724, 0.448, 0.897, 0.792)||) . If we apply||(\phi||) to every output element||(\phi(\mathcal{D}^k(a_D))||) , we get||((0.431, 0.981, 0.075, 0.277, 0.800, 0.639)||) . This is exactly the logistic orbit we saw in eqn. (10)! - When
||(a_D = a_L = 0.431||) ,, we know the logistic orbit||((0.431, 0.981, 0.075, 0.277, 0.800, 0.639)||) . If we apply||(\phi^{-1}||) to every input element before the logistic map||(\mathcal{L}(\phi^{-1}(a_L)))||) we get the dyadic orbit||((0.431, 0.862, 0.724, 0.448, 0.897, 0.792)||) .
A New Algorithm
Can we use the topological conjugacy of ||(\mathcal{D}||) and ||(\mathcal{L}||) as makeup?
While ||(\mathcal{D}||) is ugly and discontinuous, ||(\mathcal{L}||) is smooth and differentiable. We can use the logistic map as "makeup" to hide the crude dyadic operations. We want our decoder to use ||(\mathcal{L}||) instead of ||(\mathcal{D}||) . But for the encoder to glue together the bits of our dataset, we needs to be in the dyadic space so our clever bit manipulations will still work out. Here's the strategy:
||(\alpha||) with ||(np||) bits of precision that contains our entire dataset. We can now discard ||(\mathcal{X}||) entirely.
- Encoder: Work in dyadic space where bit manipulation works (use
||(\phi||) ) but output parameter in logistic space (use||(\phi^{-1}||) ) - Decoder: Work entirely in smooth logistic space using the conjugacy relationship
Encoding Algorithm: Given a datasetThe result is a single, decimal, scalar number||(\mathcal{X} = \{x_0, ..., x_n\}||) where||(x_i \in [0, 1]||) , encode the dataset into||(a_L||) :
- Transform data to dyadic coordinates:
||(z_i = \phi^{-1}(x_i) = \frac{1}{2 \pi} \arcsin( x_i )||) for||(i=1, ..., n||) - Convert each transformed number to binary with
||(p||) bits of precision:||(b_i = \text{bin}_p(z_i)||) for||(i=1, ..., n||) - Concatenate into a single binary string
||(b = b_0 \oplus ... \oplus b_n||) - Convert to decimal
||(a_D = \text{dec}(b)||) - Transform to logistic space:
||(\alpha = a_L = \phi(a_D) = \sin^2(2 \pi a_D)||)
Decoding Algorithm: Given sample index||(i||) and the encoded number||(\alpha||) , recover sample||(\tilde{x_i}||) :
- Apply the logistic map
||(\mathcal{L}||) exactly||(ip||) times||(\tilde{x}'_i = \mathcal{L}^{ip}(\alpha) = \sin^2 \Big(2^{i p} \arcsin^2(\sqrt{\alpha}) \Big)||) - Extract the first
||(p||) bits of||(\tilde{x}'_i||) 's binary representation||(b_i = \text{bin}_p(\tilde{x}'_i)||) - Covert to decimal
||(\tilde{x}_i = \text{dec}(b_i)||)
Mathematically, we can express this with a new and improved encoder ||(g||) and decoder ||(f||) :
||[
\begin{align*}
\alpha
&=
g(p, \mathcal{x}) := \phi \bigg( \text{dec} \Big( \bigoplus_{x \in \mathcal{X}} \text{bin}_p(\phi^{-1}(x)) \Big) \bigg)
\\
\tilde{x}_i
&=
f_{\alpha,p}(i)
:=
\text{dec} \Big( \text{bin}_p \Big( \mathcal{L}^{ip}(\alpha) \Big) \Big)
=
\text{dec} \Big( \text{bin}_p \Big( \sin^2 \Big(2^{ip} \arcsin(\sqrt{\alpha}) \Big) \Big) \Big)
\end{align*}
||] where ||(\oplus||) means concatenation. The decoder here is tantalizingly close to the function I promised at the start:
||[ f_{\alpha, p}(i)
=
\sin^2 \Big(
2^{x p} \arcsin^2(\sqrt{\alpha})
\Big)
||] but is still wrapped with those pesky ||(\text{dec}||) and ||(\text{bin}_p||) operations. However, something profound has happened here. We've taken the crude, discontinuous dyadic map and transformed it into something smooth and differentiable. The logistic map doesn't look like it's doing binary operations, but underneath the elegant trigonometry, it's performing exactly the same bit manipulations as its topological coungant, the dyadic map. Indeed, the makeup looks pretty great!
However, nothing is free. The cost of using the logistic map instead of the dyadic map is that our error is now ||(2 \pi||) times larger,
||[
|\tilde{x}_i - x_i | \leq \frac{2 \pi}{2^{p}} = \frac{\pi}{2^{p-1}}
||] We get this ||(2 \pi||) factor by noting that the derivative of ||(\phi||) is bounded by ||(2 \pi||) and applying the mean-value theorem. For a proof, see section 2.5 of "Real numbers, data science and chaos: How to fit any dataset with a single parameter".
How did we get ||(\mathcal{L}^{ip}(\alpha) = \sin^2 \Big(2^{i p} \arcsin^2(\sqrt{\alpha}) \Big)||) ?
We just need to perform some simple algebraic manipulation with our equations:
Code Implementation
Now comes the moment of truth. We've built up all this beautiful math about chaos theory and topological conjugacy, but can we actually code it up?
If you've been paying attention, there is one crucial implementation detail we have to worry about. If our dataset ||(\mathcal{X}||) has ||(n||) samples, each encoded with ||(p||) bits, ||(\alpha||) will contain ||(np||) bits. For ARC-AGI-2 with hundreds of tasks and high precision, this could be millions of bits. Standard computers can only handle numbers with 32 or 64 bits. How do we even store ||(\alpha||) , much less solve ARC-AGI-2 with it?
The answer is simple: we can use an arbitrary precision arithmetic library like mpmath that can represent numbers with as many bits as we want. Instead of a regular Python float, we represent ||(\alpha||) as a mpmath float with ||(np||) bits of precision. We then run the decoder with mpmath operations and convert the final result back to a regular Python float. Note: operations with arbitrary precision arithmetic libraries like mpmath tend to be significantly slower than regular floating point operations.
But mpmath gives us another gift: it actually removes the pesky ||(\text{dec}(\text{bin}_p(\cdot))||) operations from our decoder
||[ f_{\alpha, p}(i)
=
\text{dec} \Big( \text{bin}_p \Big( \mathcal{L}^{ip}(\alpha) \Big) \Big).
||] In our implementation, we use ||(\text{dec}(\text{bin}_p(\cdot))||) to truncate ||(\mathcal{L}^{ip}(\alpha)||) to exactly ||(p||) bits and then we convert ||(f_{\alpha, p}(i)||) from a ||(p||) -bit mpmath number to a Python float32. During this conversion, Python copies the first ||(p||) bits of ||(f_{\alpha, p}(i)||) and then fills the remaining bits of the Python float32 (bits ||(p+1||) through ||(32||) ) with random meaningless junk bits (assuming ||(p<=32||) ). Since our model only guarantees accuracy for the first ||(p||) bits, these random bits don't matter.
However, converting to binary and back is wildly expensive, especially when ||(\alpha||) contains millions of bits. Upon taking a closer look, we can, in fact, actually skip the entire ||(\text{dec}(\text{bin}_p(\cdot))||) step and convert ||(\mathcal{L}^{ip}(\alpha)||) directly to a Python float32. The first ||(p||) bits of ||(\mathcal{L}^{ip}(\alpha)||) still get copied correctly and bits ||(p+1||) through ||(32||) get filled with the higher-order bits of ||(\mathcal{L}^{ip}(\alpha)||) instead of random Python bits. Since our prediction only uses the first ||(p||) bits, these extra bits are irrelevant whether they come from Python or straight from our decoder ||(\mathcal{L}^{ip}(\alpha)||) . Now we can get the correct answer without the expensive ||(\text{dec}(\text{bin}_p(\cdot))||) operation since the bits ||(p+1||) through ||(32||) are disregarded and can come from anywhere. Removing ||(\text{dec}(\text{bin}_p(\cdot))||) , our decoder simplifies to exactly what we promised at the start:
||[ f_{\alpha, p}(i)
=
\mathcal{L}^{ip}(\alpha)
=
\sin^2 \Big(
2^{x p} \arcsin^2(\sqrt{\alpha})
\Big)
||] This is amazing! Usually translating math into code turns beautiful theory into ugly, complicated messes. But surprisingly, leveraging mpmath has the opposite effect and actually makes our decoder even simpler. Now let's get to the code!
Building Blocks
First, we need to import our arbitrary-precision math library, mpmath.
We need some functions to convert from binary to decimal and back. We cannot simply use python's ||([0, 1]||) .
bin function because it only converts integers to binary and we have floats in Next we need ||(\phi||) and ||(\phi^{-1}||) to go back and forth between the dyadic and logistic spaces.
We can now implement the logistic encoder
||[
\alpha
=
g(p, \mathcal{x})
=
\phi \bigg( \text{dec} \Big( \bigoplus_{x \in \mathcal{X}} \text{bin}_p(\phi^{-1}(x)) \Big) \bigg)
||] in code
We compute ||(\alpha||) in five steps using mpmath with precision set to ||(np||) bits. Crucially, step 4 produces an mpmath float with the full ||(np||) bits of precision, which we then transform in step 5 to get our final ||(np||) -bit parameter ||(\alpha||) . Next, we implement the logistic decoder
||[
\tilde{x}_i
=
f_{\alpha, p}(i)
=
\sin^2 \Big(
2^{x p} \arcsin^2(\sqrt{\alpha})
\Big)
||]
Again, we set the mpmath precision to ||(np||) bits and implement the decoder in a single line using mpmath's arbitrary-precision functions:
Sin, Arcsin, and Sqrt. That's it. Our entire encoder and decoder, the heart of our one-parameter model, is just a handful of lines and a bit of beautiful mathematics.Basic Implementation
Let's put our
logistic_encoder and logistic_decoder functions to the test and create a one-parameter model for the first task from ARC-AGI-2's public eval set.
In this task, we have 4 example input-output pairs and a question input-output pair, seperated by the vertical line. The challenge: given the 4 examples and the question input, predict the question output by discovering the underlying pattern.
Take a moment and see if you can spot it. It is quite hard.
Remember, to an LLM this is just a grid of integers in ||([0,9]||) . We've just added colors to make it easier for humans to see the pattern. From an LLM's perspective, the outputs are:
To make the one-parameter model work for ARC-AGI-2, we need to make three additions to our encoder, decoder functions:
- Supervised learning. ARC-AGI-2 is a supervised learning problem with input-output pairs
||((X,Y)||) , but our encoder can only handle unsupervised datasets||((X)||) . We simply encode the outputs||(Y||) instead of the inputs||(X||) because the outputs||(Y||) are what we actually need to memorize. - Shape handling. Our encoder works on datasets with scalar numbers, not matrices. Simple solution: flatten the matrices into long lists during encoding and then reshape back during decoding.
- Data scaling. ARC-AGI-2 uses integers 0-9, but our encoder needs values in [0,1]. We use a standard MinMaxScaler to squeeze the data into the right range during encoding and unscale them during decoding.
Let's take it step by step. First, we'll treat this as supervised learning approach by making
X the question input and y the question output.X1.shape=(1, 30, 30), y1.shape=(1, 30, 30)
X1[0, 0, :5]=array([4., 4., 1., 3., 5.])
y1[0, 0, :5]=array([7., 7., 9., 0., 0.])
Second, we transform
y from a matrix to a list of scalarsy1_flat.shape=(900,)
The question output ||(\alpha||) . Since each element requires ||(p||) bits of precision, a single task demands ||(900p||) bits, not just ||(\alpha||) requires ||(400 \cdot 900p=360,000p||) bits. It is quite easy to see why our one-parameter model may require an ||(\alpha||) with millions of bits. For now, we'll focus on a single task, which only requires ||(900p||) bits.
y starts out as a 30x30 matrix but flattens to 900 scalar elements. Crucially, this means to encode a single ARC-AGI-2 task, the one-parameter model must encode 900 individual scalar elements into p bits. This cost adds up quickly. To encode all 400 tasks in ARC-AGI-2's eval set into one Third, we transform ARC-AGI-2 inputs from ||([0, 9]||) to ||([0, 1]||) with the MinMaxScaler
scaler.range=np.float64(9.0)
y1_scaled[:5]=array([7.77777778e-01, 7.77777778e-01, 1.00000000e+00, 1.00000000e-10,
1.00000000e-10])Every entry of ||([0, 1]||) . Remember, we flatten and scale ||(\alpha||) .
y is in the proper range, y (question output) and not X (question input) because we want to encode the question output into We are ready to run our encoder and learn an ||(\alpha||) that captures all 900 elements in the first ARC-AGI-2 eval task. This is our
model.fit(). Remember, this is "training on test" as this task comes from the eval set, not the train set.full_precision=6300
len(str(alpha))=1897
We encode each sample with ||(p=7||) bits of precision. Alpha is ||(1897||) decimal digits long and ||(900 \cdot 7 = 6300||) bits long. Feel free to scroll:
alpha=0.74285114332024342325549937083636474565519009327862941959665140406783483205479218836647152731554946432856788703992188906990891731482182847524308776802281798217124682447263086292706558015080117587125137946881353134901846130213659216073847350085266801546500388056303279812229387014566738064635728388315625125612460908053813612115944371403488022330138204005465833529402443403780893627567256369296303433860649253101774272722381473011643003744092414141792080856154336852232751681761870825262961239388416686534036505209391584808776416887122876960233618275222404879749262022119613913874625326198219211198620416420976413170105387014954985331241357748594103136211408700904101070819345678337562478638731082420742643951934007189701107794645826238176372535654425768205812193697418665115645591019081599730450414681394654648740614534913490603387271476765995708667386637327422984549190564023567503974083813261600742256850890763198442464530739947767438158346711984686537123296971853036329532017930470957886617222952038244846792482431147430762987112113198332310791887702977246168036347134814764098024874588331258467793920192910184049083305781018189848769815863752005856715581710276074190641816852407672817742796240720806033022892180099812775701709359799032789650317971424349107886413845671496713331998226307819777294734024746164655829893066785266198516930143907068214149756610237486409825479704403950049986719886018007204851422317073595042856901520808205036921515904367496432996379033889021520978522808476522459195470488476590221986500425992848671776649140155585867456664879520985321530329590064710028585228533299150027623406426839690690773509222974472641729905841013913165150369477959654018728154622535075452933489985516349842376600009501513262596103419929725321662046257221998489659517783086965851309721070204164329693715232879338957676296906129816570061872627789636932222196808375062090735051961248623508890396
This single ||(\alpha||) encodes the entire task, all 900 individual elements, perfectly. This is our one-parameter model in its full glory. All we need is this alpha and we can correctly predict the question output of this task.
To decode and do
model.predict(), we must run our decoder 900 times to extract all 900 invididual scalar elements from public eval task 1 of ARC-AGI-2. We use a for loop for this in decode.y1_pred_.shape=(900,)
Now let's undo the steps from before: reshape
y1_pred_ back into a matrix and scale it back to [0, 9].y1_pred.shape=(1, 30, 30)
y1_pred[:, :5, :5]=array([[[6.68566029e+00, 6.71503815e+00, 8.97833127e+00, 9.00000000e-10,
9.00000000e-10],
[6.65239587e+00, 2.08959544e+00, 8.97833127e+00, 9.00000000e-10,
9.00000000e-10],
[6.65239587e+00, 2.08959544e+00, 8.97833127e+00, 9.00000000e-10,
9.00000000e-10],
[6.68566029e+00, 6.71503815e+00, 8.97833127e+00, 9.00000000e-10,
9.00000000e-10],
[3.67009172e+00, 3.69329225e+00, 6.62128532e+00, 9.00000000e-10,
9.00000000e-10]]])Great, now the output of the decoder is in a usual form,
y1_pred. Let's plot the results to see how well our one-parameter model actually did:
The left column shows the ground truth (correct output), while the right column displays our one-parameter model's prediction. We don't display the examples or question input here. We adjust the colors to reflect the magnitude of each error: larger deviations from the ground truth produce larger differences in color.
Look at the first row. The correct value is 7, and we predict 6.69. The next cell should be 7, and we predict 6.72. The third cell is 9, and we predict 8.98. Across the entire grid, our predictions hover near the correct values, often off by a small decimal amount.
What went wrong?
These imprecisions stem from our choice of precision parameter ||(p||) . Recall, the encoder stores each element as a ||(p||) -bit number and discards all information after ||(p||) bits. This truncation introduces a quantization error of up to ||(\frac{\pi R}{2^{p-1}} = 0.44||) where ||(p=7||) is the precision and ||(R=9||) is the range of the MinMaxScaler. (Notice that indeed all of our errors are less than 0.44.) There is nothing wrong with our encoder or decoder; this is an unavoidable consequence of finite-precision encoding.
We can, however, reduce the error by increasing the precision. Let's test this with ||(p=14||) .
When ||(p=14||) , we get every prediction perfectly correct (up to two decimal points). However, this comes at the cost of requiring a larger ||(\alpha||) , going from ||(6300||) digits to ||(6504||) digits or in binary, going from ||(900*7=63,000||) bits to ||(900*14=12,600||) bits. The larger ||(p||) is, the more accurately our encoding, but the more storage it takes up. Still, this is a pretty amazing tradeoff!
Faster Implementation
The decoder works great for a single task. But for every additional task, we have to decode another 900 elements. Decoding all 400 tasks in ARC-AGI-2 would take hours. We need a way to make this faster. Looking at our current decoder
we can accelerate this in three ways:
def logistic_decoder(alpha, full_precision, p, i):
mp.prec = full_precision
return float(Sin(2 ** (i * p) * Arcsin(Sqrt(alpha))) ** 2)
y2_pred_ = np.array([logistic_decoder(alpha2, full_precision2, p2, i) for i in range(len(y2_scaled))])
- Parallelization: Because each element is decoded independently, we can decode all elements in parallel with
multiprocessing.Pool. This speeds up the for loop overlen(y_scaled)). - Precomputation: Calculate
arcsin(sqrt(alpha))once before decoding instead of recomputing it in each decoding iteration. This eliminates repeated expensive trigonmetric and square root operations. - Adaptive precision: We currently set mpmath's precison to
full_precision = npbits at every decoding step, even though we don't need all||(np||) bits at each iteration. Instead we can use||(p(i+1)+1||) bits in the||(i||) th decoding step, drastically reducing the number of bit-operations we do in every iteration.
How does adaptive precision work? Why can we use ||(p(i+1)+1||) bits instead of ||(np||) bits in the ||(i||) th decoding step?
Each sample is encoded in Let's implement these three speedups in our code.
We modify ||(\alpha||) and set the precision to
logistic_decoder_fast to take in arcsin_sqrt_alpha instead of mp.prec = p * (i + 1) + 1 instead of mp.prec = full_precision.def fast_decode(alpha, p, y_scaled, n_workers=8):
y_idxs = list(range(len(y_scaled)))
decoder = functools.partial(logistic_decoder_fast, Arcsin(Sqrt(alpha)), p)
with multiprocessing.Pool(n_workers) as p:
y_pred = np.array(list(tqdm(p.imap(decoder, y_idxs), total=len(y_idxs), desc="Decoding")))
return y_pred
We parallelize the decoding with
multiprocessing.Pool. Note: since multiprocessing.Pool fails in notebooks we need to define fast_decode in another file and import it here. This is one of the weird quirks of parallelizing operations in a notebook and why fast_decode is displayed differently than other functions.Let's now run a speed test and compare the fast and slow decoders on 5 ARC-AGI-2 tasks with precision ||(p=7||) . First, we encode these 5 tasks into ||(\alpha||) :
lenlen(alpha)=18966
alpha=0.7775209861406958098772906114320544884971506305747653799305704150225683997037066833693169029457870017538630919380227691936713799705668441184279379903770648502916319017875848368414199715874977420187414213255944753382939870191295048156956394434729235630202594324483378612798190304822716750123038358584036576173634736125857454487302998904866696544633955333022031079393686878325382831996758758026800748158519847123973994689942920014295946408404853988439890504324931518218357711520916610934086346372176102468168500580516148117276994341498343333939166273301133363572806698539413782950565063283090178554391127972736892720146837862544879409418975290658068423819810267186009513706053031435195973928970617398391222969748941788812925505237513955188878801399493476524937317205879289743669401632266877911103820407294767619956118197726417937120286056463560342927059659243225564941486104090623121829159334361348843172763743032463515871596901730538867530291581014312981834056386485383522843041068363239004592696139444094674692264839657806717923679964139026027126843250459193580680378532734811066291292812871757799957304705959823815337981327892635955884030903952336887398337297071583616148434804992370339552251852390564653345170638429027879494893640162185555746740335701300364549366895603071509335613921022261603757192510216234788297479760745379738926413979105343703865752975290088443822457654233409320835472727119547071827341164598923414643808061642631363409867078064680443778148739652500248351753313127151296891266727601832993148940666076860261312220053449633381926994036057594926536127495919321642620459160643015776389402713240508516646662680872461180106882109916035575188062772680052927761123218010745929633559642537822589267048893038166127679828308107412347151490228081454708475263207431307255929660116229215391718713219296275124965940942264969570604653425158112995766190086742617428331894779561739932248083571327954675626661081416268502138135670526036111028909912905744513739401815865204789074622337967780543030554997349904149174852866314699055775118484307561617715476510223659879006262178277147936319561685416795550343684806292016045430825061295407629630792859378607472703865601637837921366222165768712671192582993050095992902162837409809122249239956018815349224923708459999765889318817603772647160606763515881139107465176471348966415351212897710760490417256912083079981901637762524243714446889324828803989120383830509059162495831234814783149160981763831732072833038894324971545710710008190804215644383909578819102551619474478428657329954383360916910313614352813645944159833591281540014560964897679481295864531058943314040449609060218951113512840094365270382099545227223915145586614988750802750214742576283191925349471448042434102846941962540297936541026820540718958339292143292163938823116188995239202472005635489764201396689638726576661838870926041565986890988555520913227915204944750041326290847931254424080074440665648486008496056685202787762244974149930506977416537062451368978630408639347272514083822569176408650111018195601799779042880484798985309669955587636835226642047780307652329036427222151220573734269706781154856479330513977108408939823425761197980492009203323038357441559071737728375059299834300990403980785087317309678237211414162695404774007206747165603152844307832091465282117853207409348523911168821992539322430533382367874753882421542452546543359486619420823793849027339590189927266342792439160948104213822218739216967292254361917847630601829755876830238862791259734138784172222550622543426119387383737740104020752142091281534932082362029481047655308724606308020905611082998746610739035722505549305859489094152573993473174448601713205053780138594345648815804327800554735984643409551564986962583884769310978837582786634509426107165400044691862785889096328454371614934761199944789347530960557945203896581757968231619206094854804286129511369644900152840416559920260228208747103750562551394236353982547764445613557926830701511359842450458790462806645051722115457122019565954935943198118047568582947950993335515723984720670660910463457195736063977715068710151969448833507033295741927889238677457350474957936970223139373198455841672210106870454983019760834261849916725438709284918960844415873798743398319390999053115880796908621725452782507238244596247743507550993723297438153421700713794146262460973504583532383630067979795827285166067875511097648130873061024949135163260509974603041594613882209798106696373295028893358906861988882106803070819126364653829837094143594360953758425240459676193943231858434420091544108278829125823050127216540862561291333815815917891882114933021145573952982858688405512909936162453690326347437755409384372285222522920728180045196002467872271913615004997124645275778297335044088873243025860723081818176240733540275965692542506748311577974240949543425561130152634195831681482266310865329846404794228984099887003817431935952514958013390643244554627487560307898156274617073857788657582135184601159321290298136722212914863282521260925309643762584662601840201819807347006690730305999309634680059577898115496126040679427026812123499278047987428302883924674410228668173033915416437624288673053432461493434502495570269602833977336669130858166071043116008590160705000907740494697228003075793173013469713164690705004758922728278057227813079498186314023595039759358933422810450331341929311397713658207126691717677878335691177740434601522409130473486822737744612114741932888661749818185189238420974479331886319996130269614670102707039743250659139394447063467707469173685051966719041317024841955178272005799125504626971165859221278224140789326815037008758752489832282928439129560734950215147836308464173977016493289431405436356113395726917126954629133780126458384180129125870730631790722213719524471650861933345488700197431584495034291164118183566718990705708618001597451435020605249210871153129469121553418665308218746303551977820550577365886826535546921752431916056854281227303128784141611869413342368941925157667019406639000655443698705062327976454674007908325531797777200170920718038006559703695480917150127331711148972005103941187410934062470011960606382899852996276308842931818932722548008934316220166413508243656096974843774364568122483054895698919657980527085047017955240561142364808715073571082857799779858575487359739043691565263502161751494179198153302417393450562555972715672624886656879054452611236741488283285013164387141911737214926502144529674841831621740179150134058517779580803184992696478946162737459195526851953673941297538659611911379347500132340340702740471194876444526320453452499856139742662560094705035086488653325646359022974174953413628200886706671043969336656317225914181364868901633383304078875933012989250802781871969334836594758989348182684951736528693850473710969158026082556353263307451145608657688600685809112177946671294246466342459457945913111066747879136806639609689535989475840818938019636996378169519096854574144097135750709680902398937265001096189703148628075292051950637001754977572533951047677728735147314344923802664660278417992389976658773139193698611163219207643945926105993312332154666451751609619860448787473451627755128597988133402844770754951602788283926436193227405030272703073350738663368466123231024735066413185378009697506432103046711726928784833183015418127803485098809834060658656672445633190371507424186454710082217830348745096264952263314055502828054921481164782000783564074473076570693407037183180543231132840365997133898047395013754892577914081407533813881172679102848671760495807653540811325430339562901811492777947612236375451372145148164960497377110680468130432464875760730990473465003392827815368854434033954452224275723629236439045497646649994081824885941319834251627067327245515012422953757567382203061183863112361111893890291919637355110220529675770919406083597586574046920022845441780737467078746067147877540404388982509125620206510518718707762495857617482714825638435028127602077178815441078582045963184452018853571900407232257252004829221628395104072094286315512125866132012851977810429320771571957299454195194595557915818810860815675859338124192816468894096999189626512638162888567663637203369045582388563300461049907279460961646711271724991937034945592240898477673655851148730841113004356785562754125744146775609740791503840899731340008588427699285386009807593832369898883352714841952100958880326076506326879504734739836326158733380822821460104359732672281040247195637417671381461805214241292475142582270469930897253108183290122645884570608221962423855169471267184539959469062613238256338061885654017636466028285028211176436508284000057603667518410702749449032836582852514803909637360163501138858506984897340750715109637047424344170143235428737201550738213842425306603074391112565729451315888550109113853794403072998784154700761828962405192936610674055342279393193931162489283021130993435904552079317731195711479474049063278927305922538253391280088530095689942541814315913098688630969472598984885361849926286108283841804403821798821352441355187675591787478055955886250722870076339490021562692799999052874997412404247312058980214677096061133602291142979890586512965083997824185290295771755155762423675208778327485065146662423891997290725994142719803206390552938641959054670221845602104808500585585289307443162950135568700466953728405562584361183060033978531052771761066380691086850073298216242477319171400120183774473485697544593101159398867818882639914541884690394356014603014659066486109218377985728939371843198308178326299996092450806502890673857403089963766135712693282730463675931698492416300423282058723359156644631333448448340697223426696872258881721523971382531615016457465305710384454752809720361813581207995616373568384574167782652752204951223841342035061501761824165337938683057553650929224610772007851180863780168450083085530554934256124770481106777531434919545708215770992308891501008788817510257566768452278635638544856844862248010112226721524244991390564535578276005679974889418963597452669612078826563311402943533566958418491993579987624057143899026279343725718373806699560661821155986866363530584248797302576369248401744121093742220319052573124028368779737881252195384367600575029531550132019526720645425648230264982371404783207345026262205289867814357240255204102309201009094822061878286019588820059435226845337338338195731746136511045319677167974752732979981394330493469051275770643324396708641560222504670346537379414134244846440426320846188139732750026803337204643522927109212984412900419202866973133102737960830175840977881186118617708232780549455505864313612778249906509228319904758017922651076395740925005069280909637032790900194546397787135618577054971387292946717935268351101241229106868710026586692336371910643239066995087944618601646217219772692349313491925504739911663910835045436664769333263746473557686203872849274715402355003475306958405789247876579931147643818560151607141775233512706365389445209562354206260586029901898056368414866986292207671056885513784467667755153635810970280782474158549412571774780685806536764141420896307498472627732115881433828624027136465381023887243657300005521316506711271628339699480933658439575690433929469186584649834093030241102592314188299333456813647915611320190024202255158614395996799051651051003488873026208178811390683166558800965786851984241023209089025095929241695958402756728784442829224515101146173301504775789699100714539899234834542547693210649649440446431733311057469665064126385864309965973900944926599516608912173391096434371614135658668864080368656056329520099021194384100028294934693258820251431900108339816862173872817244576227658695298218690199894396060581171913595500992568483053761747329190203864367504040818300212993713981312061175816344911838375469350226635823857449879218884337417555239502220561427011462879075545434783190760463150361855392828561877842600863632024683289546426805997798571888151961089455987902060256305703943736784681076578348879473869131473498102955029482868267125039707136405914154768111955242891908706226165779043441390518661993410404715075409585226522274510139034738124716753149372244374139848826974994131564040488707217377783356376906545215460124459915659465635418948823574242955704101556175915573294875082106690280824254219553453194353586269031861592427226655608752537062091711086666662341296958768058171514692870794518706245258471776885591363596539274130280565543503797633003703166917452385287785346328685521326584796327450677640921847544970296403938491437935868400248896166385142627506491122191846428712329475154571787093589194983775973425762588490515207989211595754456962961525871108894632589240812211422645065115401511058885452531063569042929991619303340341981216704213056227399081764218538830752577513041528894826989994892953424269544262252697052432782335811493025871623462472955650679221376534622279455279706311481682293454980405191115348124512457028690914919674646533250848344867559505336778537023761816558993643526349569012334529159388877064422774208935342017743547955353528209170352922322120019250745267134381434742189097874320186156945106492082777020364322660889498947022865152519709016283399182824072682148524860914423997266954323791902293327962576537220555797400215343633771628428021785572835934035541822056893239692874978244022026912095803580274378894627399877979192100531293387881193919122670825844500753204106954048280736983951012160845775591384078721443665892058456059272282764951208859714847525880595921047021697353976869159422819759654015687986622259791340366503397550292522727610696406302742388305180221370322521220164810429339928397965133219292460688138648558372049319005032635009800358093328668306246686374456876793518988238953783423189596805987163077643892428514572504640162276019253105291415150986346170083574901212283860453553631841116428677110820207900300505718178409680061445597729468761462521349305896781068093311498328182146740248240268660808104272177923766912732504746219681968761883102818834144623617660183795210537762050511314928250674890301812928271589291263957955218213095253391873881245054458366742159300431727984262032882553308658376363171116339435462184874574886245836173145412480792109961196353768781899477226446478207683775810089924501629762596246589485842402424748914562117338189287460171190798590653729613898143754724123362193121821908611423086284670776701172546196314721692096247608046857198776245852472382030631422717540199264621661629602664586239675077744067384005818604114602909954316247960549125063361523714860658947041331219566726627023579298379377420965861461109650225534722353903346552912545767356921857793022713892829621949730908910953422162040030269801512664508118441790334558712385077698631557181945102827722951112157667797329516361280640593852025015174678410391593827403824623602587757546523223849244375580463902167915498142187250248435439116329664401474294342072884489971226027652844439435744862477798911445908752336295459190943326166314583647034842774606012966598250341487924352831859856990825050386225745145522535027188870386808569465137865946575449999213767375410422858671687510118379181750396854070748913306849552129797048336503819565045083401446173369389819624286925099427793752430354473033057440033816140357839628318527116255618510633333998819867597317725950712838120802265854192882721448892612176064615232362858476633919584065329801042971941938007318761212229340124827241344176813304763328552271409655417492858124057118699750510292446735765795564976365217164195347712663470236452310702144583451434361781743486412568471115168854801215599668306964593453148111585219392335998138313134600021897277082429826991106113429474377607360347949573357276669998959985924192698632429420971177922942423442358272765382763057378511731151903417914098188130969452955607523877703756471174901064691100052011598083472757203543235758262496686387611218474923923165654713577987155885134103630022219674876336037442240959619837402379886585618014652319786128768404542866765719876636115109142428227951474496441985829009536055319232514540072236249405030280186355982420161660404651470986504999636496495410529945923158193176563933620706573122136613514595713377568928994961821877537091502853637382647867596889775981987270320916758647051524998598395438116716225127088729114515573996281429822261366420555798563716047088609365339919152111561686778921446276104969887483937242793821073096271820269516665622433541530107278032710236056366808288668776199540608947172652824486017055980662911415962118089931962362042336464107622064045083479400645362874946983488314725755936646983096568088651038566734780887003114139266557641530560191194151509753141412524711014434422823723105529063648169458305938276389413928506041813601867683616920681916200502126081562813801207780349357663770921323259690669709004620767619437607768486329180953612159051115559182466361138477090159199699856443961477461148704244598382442514021210236738525480355281238464358183003844441485439854688637295729883954485731815164801928790767190091606038924139325843357451777540141578833887262904163270557116823529776792925396348198873428209523530962273155333837310200394356066024559259673838308289831751156047972298022320618750852473305676430062324258297283358920626458876632563656869090133831776915816387631040847534628063622194189605388242563393775563232239116402575693577978073262451856970066901283979952351938727744751356958866230144004104679739177507693944129226413146306003755711482643826099215953563217027346740129439024773231341152172387047790899515011162416048619925134765284270487812208315159466727018845745237244220117679977443938523884610382384380336548282556241090134418956829267606761314149677571239580662386518988108574182666370875286869780958815403991125347008692655477982801863532855438304737904194848272669969578633615551471501549615472411097391559210093133894803708188851116617415420730454477451956568659857962282116666148182987035845068216643552464723918098563719937585615551156139606957810916440617498547752296239077096166183693599598808572300578097960597821219315343373084063573235960444374798636438213334181591873311551025177272045072867825680311674749051897617899821461418749369333621137114583933680106046401271714440890168441735027176958204970234547028610269506310756811844828313310184266157950178887231070233567097250223166265286166347023231463248982874320074649899495880671762849495283036165636848830994143622255789184784594070456469282817769764839066373418110399375816877995189298473869844729741266063320168869980596229458247887218696090268324791009667313801609060734364769074845780577024193200433427560946524515147560195471004876612141393398526437826643769937429970823517322643896752147367060561748855613294130411511619994479892670500644721650560931006430129737535825833973522436855378577805454152567375343179691617238371188860253122955832257913572721337757907999151004757711367623678425831237259455549070865843892126371174923370998260175002779697303714886663729917572406679417447603983416035057686508983527128676976615632054552575624116681852008314561766749072967529611123205822403757110511534853445302606836913084815545674679193376133150056756342007702381
Both the original decoder and the fast decoder use the same ||(\alpha||) , which has 543 digits. Now let's run the decoders.
Decoding Took 125.97444974980317 secs
Fast Decoding Took 7.199744374956936 secs
The original decoder runs in 35.6 seconds and the fast decoder in 2.7 seconds. This is a 13x speedup on my Mac M1 Pro! Since we encode 5 tasks, each with 900 samples, we've decoded a total of ||(5 \cdot 900=4500||) individual tasks.
Due to adaptive precision, the fast decoder may produce slightly different values after the first ||(p||) bits compared to the regular decoder. We verify that both decoders remain within the theoretical error tolerance of each other.
This is great! We now have a fast decoder implementation that can handle multiple tasks!
Final Implementation
We are now ready to create the final implementation of the one-parameter model. (Again, because
multiprocessing.Pool fails in notebooks we define OneParameterModel in another file and import it here.)class OneParameterModel:
def __init__(self, precision:int=8, n_workers:int=8):
self.precision = precision # number of bits per sample
self.n_workers = n_workers
self.scaler = MinMaxScaler()
@Timing("fit: ")
def fit(self, X:np.ndarray, y:np.ndarray|None=None):
# if the dataset is unsupervised, treat X like the y
if y is None: y = X
# store shape/size of a single label y
self.y_shape = y.shape[1:] # pylint: disable=attribute-defined-outside-init
self.y_size = np.array(self.y_shape, dtype=int).prod().item() # pylint: disable=attribute-defined-outside-init
# scale labels to be in [0, 1]
y_scaled = self.scaler.fit_transform(y.flatten())
assert 0 <= y_scaled.min() <= y_scaled.max() <= 1, f"y_scaled must be in [0, 1] but got [{y_scaled.min()}, {y_scaled.max()}]"
# compute alpha with arbitrary floating-point precision
self.full_precision = y.size * self.precision # pylint: disable=attribute-defined-outside-init
self.alpha = logistic_encoder(y_scaled, self.precision, self.full_precision) # pylint: disable=attribute-defined-outside-init
return self
@Timing("predict: ")
def predict(self, idxs:np.ndarray, fast:bool=True):
full_idxs = (np.tile(np.arange(self.y_size), (len(idxs), 1)) + idxs[:, None] * self.y_size).flatten().tolist()
# choose the fast or slow logistic decoder
if fast: decoder = functools.partial(logistic_decoder_fast, Arcsin(Sqrt(self.alpha)), self.precision)
else: decoder = functools.partial(logistic_decoder, self.alpha, self.full_precision, self.precision)
# run the decoder sequentially/in parallel and then unscale+reshape y_pred
if self.n_workers == 0:
y_pred = np.array([decoder(idx) for idx in tqdm(full_idxs)])
else:
with multiprocessing.Pool(self.n_workers) as p:
y_pred = np.array(list(tqdm(p.imap(decoder, full_idxs), total=len(full_idxs), desc="Decoding")))
return self.scaler.inverse_transform(y_pred).reshape((-1, *self.y_shape))
@Timing("verify: ")
def verify(self, y: np.ndarray, y_pred: np.ndarray):
# check logistic decode error is within theoretical bounds (section 2.5 https://arxiv.org/pdf/1904.12320)
# |y - y_pred| <= π / 2^(p-1) when y, y_pred ∈ [0, 1]
# |y - y_pred| <= (π / 2^(p-1)) * range when y, y_pred ∈ [min, max], range = max - min
# multiply the tolerance by the scaler range to account for scaling
tolerance = np.pi / 2 ** (self.precision - 1) * self.scaler.range
np.testing.assert_allclose(y_pred, y, atol=tolerance, rtol=0)
The code is quite simple and looks like a standard scikit-learn ML model:
model.fitruns the encoder. It also scales and reshapes the data.model.predictruns the (fast) decoder. It runs the decoder in parallel and reverses the data scaling, reshaping.model.verfiychecks that the outputted predictions are within the theoretical error bounds we derived.
OneParameterModel itself is ~50 lines of code and the math functions it uses are probably another ~50 lines of code (phi, phi_inverse, decimal_to_binary, binary_to_decimal, logistic_encoder, and logistic_decoder). Only around 100 lines to get a perfect score on ARC-AGI-2!<public.src.model.OneParameterModel object at 0x11b81e9d0>
Conclusion
"With four parameters I can fit an elephant, and with five I can make him wiggle his trunk." - John von Neumann
Other uses for the one-parameter model
We've built a model that achieves 100% on ARC-AGI-2 with one parameter. Indeed, this technique is quite powerful and can be applied to tons of other datasets, achieving perfect accuracy every time.
For instance, we can encode animal shapes with different values of ||(\alpha||)
We can find an ||(\alpha||) that perfectly predicts the fluctuations of the S&P 500 for ~6 months with
alpha = 0.9186525008673170697061215177743819472103574383504939864690954692792184358812098296063847317394708021665491910117472119056871470143410398692872752461892785029829514157709738923288994766865216570536672099485574178884250989741343121
And we can even find values of ||(\alpha||) that generate parts of the famous CIFAR-10 dataset
This technique is incredibly verstile, able to achieve perfect acuracy across tons of different domains.
To the critics
Yet at the same time, the one-parameter model is incredibly brittle. It exists as a crude hack, directly encoding the train set into a single parameter. Even simply shuffling the dataset will cause your model to break down. The decoder depends on the index ||(i||) , not the sample ||(x_i||) . It should be abundantly clear that the one-parameter model has no ability to generalize whatsoever. It would get a 0% on the private, heldout test set of ARCI-AGI-2.
Two quick technical notes to the critics.
For the complexity theorists, yes, this is cheating. We've violated the fundamental assumption of bounded-precision arithmetic. Most complexity problems assume we operate on a machine with an ||(\omega||) -bit word-size. However, my one-parameter model assumes we can operate on a machine with infinite bit word-size.
For the deep learning theorists, of course our one-parameter model can memorize any dataset. Our decoder contains ||(\sin||) which has an infinite VC dimension, i.e. an unbounded hypothesis class, and is therefore infinitely expressive. It can learn anything. What is interesting about the one-parameter model is that it offers a tangible construction, not merely a claim of existence, for learning any dataset.
Lessons
A couple of key takeaways here.
Intelligence is not parameter count.
The existence of such a simple equation
||[
f_{\alpha, p}(i)
=
\sin^2 \Big(
2^{i p} \arcsin(\sqrt{\alpha})
\Big)
||] with such powerful expressivity deomonstrates that model complexity cannot be determined by counting parameters alone. The one-parameter model exploits a often-overlooked fact: a single real-valued parameter can encode an unbounded amount of information by hiding complexity in its digits rather than in parameter count. In other words, don't automatically assume that a bigger model is a smarter model. Parameter count can be a poor proxy for intelligence.
Intelligence is compression.
To compress data, you must find regularities in it and finding regularities fundamentally requires intelligent pattern matching. If intelligence is compression, then our one-parameter model has all the intelligence of a phonebook. It achieves zero compression and is just a nice lookup table. It cannot discover patterns or extract structure. The one-parameter model simply stores the raw data and uses the precision ||(p||) as a tunable recovery knob.
Real compression requires understanding. If you want to measure the complexity and expressivity of machine learning models, measure their compression. Use minimum description length or Kolmogorov complexity. These techniques capture whether a model has actually learned the underlying patterns. They cut through the illusion of parameter counts and reveal what the model truly understands.
Prof. Albert Gu's paper ARC-AGI without pretraining actually does this right. They used a general-purpose compression algorithm to solve ARC-AGI without training on the test set. That's the real deal. Our one-parameter model is a degenerate version of the same idea.
Training on Test
Our one-parameter model takes the idea of "training on test" to the extreme: it encodes the entire test set directly into ||(\alpha||) , achieving 100% accuracy while learning nothing. The one-parameter model is utterly impractical and, frankly, an absurd hack. But that's precisely the point: it is absurd to train on the test set just to get to the top of a leaderboard.
Yet this is exactly what occurs in the AI community.
Top AI labs quietly train on their test sets. The one-parameter model does it proudly.
It is rumoured these labs have entire teams who generate synthetic dataset for the sole purpose of succeeding on a specific benchmark. This is overfitting taken to the extreme.
The ARC-AGI Benchmark
ARC-AGI was intentionally designed to resist overfitting. It uses a private test set for official scoring, making training on test impossible. (Our one-parameter model only trained on the public eval set.) Yet even ARC-AGI's organizers have raised concerns about test contamination creeping into modern AI development.
Yet there is a more fundamental point: many of the amazing approaches to the ARC-AGI competition seem to overfit to ARC-AGI itself. Researchers generate synthetic data specific to ARC-AGI and create abstractions unique to the grid structure of the competition. I doubht many of their techniques would generalize to other reasoning problems. How many of the innovative solutions to ARC-AGI have inspired downstream improvements in LLMs or other modes of intelligence? I hope these techniques prove to be good for more than just ARC-AGI's delightful puzzles and drive broader innovation the field of AI.
Final Thoughts
To close, I’ll return to the line we began with:
"When a measure becomes a target, it ceases to be a good measure" - Charles GoodhartIndeed, the one-parameter model serves as a reductio ad absurdum of what happens when metrics become the goal rather than the guide.
If you liked this or want to chat, reach out! I always love talking to people working in interesting problems.
To cite this blog post
@online{Turok2025ARCAGI,
author = {Ethan Turok},
title = {I built a one-parameter model that gets 100% on ARC-AGI-2},
year = {2025},
url = {https://eitanturok.github.io/one-parameter-model/},
}
I built a one-parameter model that gets 100% on ARC-AGI-2
Intro
ARC-AGI
The HRM Drama
Chaos Theory
The Dyadic Map As An ML Model
A Worked Example
The Algorithm
Applying Some Makeup
Logistic Map
A New Algorithm
Code Implementation
Building Blocks
Basic Implementation
Faster Implementation
Final Implementation
Conclusion
Other uses for the one-parameter model
To the critics
Lessons
